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LOAD BALANCING IN A NETWORK 

5 CROSS-REFERENCES TO RELATED APPLICATIONS 

The present application claims priority to U.S. Provisional Application No. 
60/188,142 filed March 10, 2000 (Attorney Docket No. 4835-US); and U.S. 
Patent Application No. 09/779,071 filed February 7, 2001 (Attorney Docket No. 
10 UDN0005), commonly owned, and hereby incorporated by reference for all 
purposes. 

BACKGROUND OF THE INVENTION 

15 

TECHNICAL FIELD 

The invention relates to network traffic management in a computer environment. 
More particularly, the invention relates to load balancing and network traffic 
20 management among Web servers or caches in a computer environment. 

DESCRIPTION OF THE PRIOR ART 

There are a number of load balancing and traffic management products available 
25 for performing load balancing/traffic management among a cluster of web 
s • vers. However, none have the ability to schedule both HTTP and HTTPS 
(SSL) traffic persistently based on user sessions (cookies or session identifiers in 
HTTP GET requests). 

30 The majority of the approaches suffer from having a bottleneck because they 
require all traffic coming in and out of the cluster to go through a single machine. 
The single machine limits the amount of throughput available in the cluster. 
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* man y apples **^£ZZ™"^ 
of web server caches. None can fully utfee HTTP MP* 
performing URL based scheduling across multiple machrnes. 

content. 

• «, ™ ThPre are those where all traffic in 
n6 Load balancers come in two basrc flavor. Jhereare ^ ^ 

and out o, me site goes through a s,ng e <~ ^ back through the 
Data flows in through a schedu,er, then to a Web sen/er, 
scheduler and out to the client. 

- = =. 

from the Web server. 

former — dcs no, require ^ J^J^^. 
25 Custer. The latter solution requires specral software on ft. sew 
Because of this, both have advantages and disadvantages. 

^er since traffic going out does,, go throug of 

through the scheduler. 
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The four way data flow suffers from a throughput bottleneck, but isn't hard to 
implement because a box is placed in the network, configured, and it does its 
work. This approach does not require any software on the servers in the cluster. 

Persistence is the ability to keep an individual user session tied to a single 
machine. Almost all load balancers have various policies for scheduling and 
maintaining persistence. All packets from the individual user will be sent to the 
machine that he is persistent with. That way, a machine can maintain the state 
of the user since the user is always scheduled to the same machine. 



Most load balancing systems allow scheduling based on information about the 
client (IP address) or content contained in a request (cookie, content requests, 
etc.). However, since these systems are normally based on simple routing 
techniques, they tend to fail when it comes to dealing with requests that are 
1 5 encrypted since they do not have the ability to decrypt the request. 

It would be advantageous to provide a decrypting load balancing array system p^^^ 
that provides load balancing and network management across Web servers and 
bypasses the single server bottleneck in a computer network. It would further be 
20 advantageous to provide a decrypting load balancing array system that decrypts 
SSL requests and performs SSL session scheduling. 



25 



SUMMARY OF THE HNVENT1QM " ' ' 



The invention provides a decrypting load balancing array system. The system 
provides load balancing and network management of traffic through a cluster of 
servers that avoids the typical single server bottleneck. In addition, the invention 
allows the decryption/encryption and session scheduling of SSL requests through 
30 independent load balancing servers. 

A preferred embodiment of the invention uses a Pentaflow approach to network 
traffic management. Pentaflow extends across an array of Decrypting Load 
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Other aspects and advantages of the invention - become apparent from the 
following detailed description in combination with the accompanying drawngs, 
illustrating, by way of example, the principles of the invent™. 
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Balancing Array (DLBA, servers sitting in front of back end Web server, One of 
,he DLBA sewers acts as a scheduler for the array. All incoming requests are 
routed through the scheduler. 

5 The scheduler routes and load balances the traffic «o the other DLBA servers 
(including **) in the array. Each DLBA server routes and load ba^ces » 
Looming request packets to the appropriate back end Web server, Responses 
"quels from the back end Web servers are sen, back to the DLBA sewer 
which forwards the response directly to the requesting client. 

10 each DLBA server has the ability to decrypt SSL sessions In a distributed fashion 
and then schedule sessions to back end Web servers based on a cook,e or 
session ID. SSL packets are decrypted in me DLBA server before be,ng routed 
t0 a back end Web server. This allows the DLBA server to schedule SSL 

15 sessions to back end Web sewers based on a cookie or session ID. Response 
packets are encrypted by the DLBA server before being forwarded to the Cent. 

The invention also uses cookie injection to map a client to a specif* back end 
Web server, .n addition, any DLBA server in the array is capable o, «ak,ng over 
20 the scheduler functionality in case of scheduler failure. 

URL based scheduling and hash scheduling of request packets witt. keepalive 

connections is easily performed due to the invention's architecture. gppg 
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rriff DESCRIP ™N OF THE DRAWINGS 

Fig . 1 is a block schematic diagram of a preferred embodiment of the invention's 
Pentaflow packet flow according to the invention; 
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Fig. 2 is a block schematic diagram of an example of a preferred embodiment of 
the invention's Decrypting Load Balancing Array cluster according to the 
invention; 

5 

Fig. 3 is a block schematic diagram of a logical viewpoint of a preferred 
embodiment of the invention's Decrypting Load Balancing Array cluster 
according to the invention; 

10 Fig. 4 is a diagram of an example of a Client IP Hash Table according to the 
invention; 

Fig. 5 is a diagram of an example of a DLBA Server Status Table according to 
the invention; 

15 

Fig. 6 is a block schematic diagram of a DLBA server interface according to the 
invention; 

Fig. 7 is a diagram of an example of a Web Server Status Table according to the 
20 invention; 

Fig. 8 is a diagram of an example of a Summary Web Server Status Table 
according to the invention; and 

25 Fig. 9 is a block schematic diagram of a task-level viewpoint of a preferred 
embodiment of the invention according to the invention. 
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„ in a deotvpting load balancing array system in a 
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computer enwonment. A system ^ ^ ^ 

balancing and network management o. Mlto »» ^ 

avoids the MM T s SL enuests through 

decryption/encryption and session schedule of SSL reoue 

10 independent load balancing servers. 

■h« a Decrvpting Load Balancing Array (DLBA) system. A 

traffic to the back end Web servers. 

A DLBA system can be used to replace — 

««^*r^ttl-* - UP or .epioy 
performs decryption, removing the need tor 

servers or hardware to support SSL. 
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High-level Benefits of the DLBA 
25 Some of the features of the DLBA are as follows: 

. Single virtual IP address for many Web servers. 
. Pentaflow- 5 way pipelined data flow. 

. cookie based persistence/session id scheduling for both HTTPS (SSL) 
. Fully redundant schedulers for scheduler failure protection. 
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o Unique hash group based persistence tables to keep persistence tables 
manageable. 

The first item is supported by all local load balancers. All load balancers publish 
5 a Virtual TP (VIP) address to the world and then have many machines which can 
take traffic for that IP address. 

Referring to Fig. 1, the Pentaflow feature is shown. The five-way pipelined data 
flow is a unique feature of the invention. What Pentaflow does is cause the flow 
10 of traffic coming from a client across the Internet or network 104 to be scheduled 
105 to a DLBA server A 101, DLBA server A 101 schedules the traffic 106 to 
DLBA server B 102. DLBA server B 102 load balances traffic 107 to Web server 
C 103. Web server C 103 responds 108 back to DLBA server B 102, which 
sends the response 109 back to the client 104. 

15 

Pentaflow causes traffic in and out of the system to go through different 
machines. All traffic coming into the system will come in via route 1 105. 
However from route 1 105, the traffic could be scheduled to any DLBA server in 
the cluster. In Fig. 1 only one other DLBA server 102 is shown but, in practice, 
20 there will be an array of DLBA servers to which traffic may be scheduled to any 
one server. Here, the DLBA server 102 that the traffic is scheduled to make a 
load balancing decision to decide which back end Web server to communicate 
with. It then communicates with the Web server 103 and returns a response to 
the client. 

25 

The invention can scale to almost any amount of throughput. The only limitation 
is the scheduler's ability to handle incoming traffic. Outgoing traffic is balanced 
among many DLBA servers. The servers utilize the full bandwidth possible for 
their connections because there is no bottleneck in the array. Each individual 
30 server passes connections through the switch at the maximum possible 
bandwidth. 
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The user will always be scheduled to the same machine. No other existing single 
solution can provide this. 

The DLBA also allows URL based scheduling, including hash scheduling, for 
scheduling URL requests to caches where a single connection can be scheduling 
to multiple back end machines and a keep-alive connection with the client can 
still be maintained. Many load balancers currently support basic URL based 
scheduling with their scheduling algorithms - DLBA URL based scheduling is 
different. 

A DLBA server establishes a connection with a client and the client keeps a 
connection alive with the DLBA server. When requests come in, the DLBA 
server schedules the requests to different back end Web servers via different 
connections and replies to the client using a single connection. 



No other load balancer can support this because they do not maintain a separate 
connection with the client. To perform URL based scheduling using these 
approaches, keep-alive support must be turned off on all the load balancing 
servers. Some servers (e.g., Microsoft IIS) do not support turning off keep-alive 
20 with HTTP connections, thereby causing their URL based scheduling to fail. 

Another advantage of the DLBA is that schedulers are fully redundant in case of 
failure. Any one of the boxes in the array can become a scheduler at any time in 
case the main scheduler fails. The scheduler is basically a simple router that 

25 persistently maps requests based on an internal table. Also, the scheduler boxes 
can also be DLBA servers and can schedule connections to themselves. The 
invention is different from many of the other load balancers where two load 
balancing boxes are purchased and one box just sits idle and only becomes 
active when the other fails. In a DLBA system, as long as a machine is 

30 functioning, it will actively server some function in the array. 

The DLBA server maintains persistent connections in all its paths when required 
and a special algorithm, hash group based persistence, is used to keep the 
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Summary of Advantages 
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Speedera Content Deliver Netwo* and then ^ e ^ f r 
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domain name to give the Speedera u. 
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These are just some of the features the invention could add to the system 
because it controls the entire flow of traffic from the DNS level to the HTTP 
delivery level and is able to parse or change things in any way along the flow. 

5 DLBA Scheduler Architecture 

All traffic into the array comes in through a box known as the DLBA scheduler. 
The DLBA scheduler is a DLBA server that has ARPed for the VIP address. 

10 With respect to Fig. 2, all DLBA servers 202, 203, 204 in the array 205 have the 
VIP address assigned on a loopback interface (or Ethernet alias if possible), but 
only one machine in the cluster 205 ARPs for the VIP, causing the switch 201 to 
send all traffic for the VIP to that machine. All machines have the VIP address 
on an interface (loopback or physical) so they can send traffic back out to the 

15 client from the VIP address and so they can take over as the scheduler in case of 
a scheduler failure. Any machine in the cluster 205 has the ability to become a 
scheduler at any time in case the current scheduler fails. 

The job of the DLBA scheduler is to route/load balance incoming traffic among 
20 the DLBA servers in the cluster in a persistent manner. It does this using routing 
by changing the incoming packet's MAC address to another machine in the 
cluster and then forwarding the packet, unchanged, back out through the switch. 
In this vein, the DLBA scheduler is, essentially, a router. 

25 The scheduler may also simply process the packet itself instead of forwarding it 
out to another machine in the cluster if the load balancing decision determines 
that the scheduler machine itself is the best machine to process the connection. 

The scheduler machine does not perform the three way handshake with the 
30 client, it simply forwards on all incoming packets to itself or another DLBA server 
in the cluster for processing. Those machines will respond directly back to the 
client. In this way, all traffic coming into the site comes in through the scheduler 
and all traffic going out is load balanced among the DLBA servers in the cluster. 

11 
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205. The DLBA scheduler is the DLBA serve 
array 205 comes in through that server. 

5 irtniral Pentaflow DLBA array diagram is shown. Fig. 

Referring to Fig. 3, a more iog,cai Pentafio ^ ^ ft> 

3 shows the same array configur .dor , - F«. 2, . ^ ^ ^ 

reiationship o. the DLBA soheduier 302 to the - ^ 
The switch 301 sits iogicaiiy above .Ighou, the array 

10 DLBA scheduler 302 which routes and ioad balances 

303. 

Configuration 
program. 

, „ DLBA scheduier code is .erne, 

single ioctl() should suffice. 
Internal Operation 

25 «*^^-rr^^^- 

30 h eduier routes them by changmg the MAC ^d ^ 
forwards the packets. » respect to F,g ^packets 
aC.ien.iP/Por..oDLBASe,verHashTab,e401. 

30 u .h= ni BA Server number 402 is found, the 

herring to 4 and 5 ^ ^ ^ ^ ^ MAC 

scheduler can consult the DLBA berve 
address 502 it should route the packet to. 
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When a packet comes in, the Client IP and ephemeral port of the packet are 
hashed to a bucket in the Client IP/Port to DLBA Server Hash Table 401. The 
table contains the DLBA server number 402 to route the connection to. The size 
5 of the hash table is configurable but typically only at the time the array is started. 
It normally is not changed on a running cluster. 

The incoming client IP and ephemeral port could be hashed as simply as this: 



10 bucket = (32 bit client IP address + 16 bit client port) % size of hash table 

How does the Client IP Hash Table get filled? Initially, the hash table is empty 
with a unsigned -1 set for each DLBA server #. When a client connection comes 
in, if the DLBA server number it hashes to is -1 , a load balancing decision will 
15 need to be made. The DLBA Server Status table is consulted to determine which 
DLBA server is best suited to receive the new connection. This can be as simple 
as finding the machine with the lowest # IP Groups assigned. 



Referring again to Figs. 4 and 5, the # IP Groups Assigned column 503 in the 
20 DLBA Server Status Table 501 shows how many IP groups (hash entries) have 
been assigned to this server. To evenly distribute traffic, each time the server is 
added as an entry into the Client IP Table 401, the # IP Groups 503 assigned 
value for that server should be incremented. The weight 51 1 can also be used to 
determine how many IP groups should be assigned to a server as well as the 



When a DLBA server fails, the scheduler should assign all entries in the Client IP 
table that map to that server to unsigned -1 . This will cause the system to route 
around the failed box and assign new servers to the failed connections in a load 
30 balanced manner. 

Entries in the Client IP Hash Table cannot simply be reassigned to a server when 
it is added to the cluster. This is because there may be active connections that 




25 



number of hits 505 or # open connections 506. 
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The DLBA Server Number 508, MAC Address 502, Physical IP Address 509, 
Enabled 510, and Weight 51 1 columns come from the configuration file. 

5 Scheduler Failover 

A failover daemon runs on each DLBA server machine and its job is to contact 
the scheduler at a regular interval to obtain a current client IP table and to see if 
the scheduler has failed. It determines failure of the scheduler by a simple ping 
10 test to the box to determine if the kernel is running properly. If the ping 
succeeds, the kernel is running and it is assumed that the machine is running. It 
does not use whether the current client IP table could be obtained as a 
determination of whether the scheduler is up since that may be prone to failure 
itself. 

15 

The client IP table can have a version number on the scheduler and on each 
server so the whole table does not need to be sent at each interval if it has not 
changed. Only the server numbers to hash buckets needs to be versioned and 
synchronized across the machines, the last access times do not need to be. 
20 When failover occurs, the last access time is set to the current time for all entries 
in the table. 

The DLBA Server Status Table does not need to be synchronized since it is 
created from the configuration file and each DLBA server is pushed the 
25 configuration file during configuration. The dynamic columns of the DLBA Server 
Status Table do not relate to existing sessions in process. 

A DLBA server determines the active scheduler by checking its ARP table for the 
VIP address. The ARP table contains the MAC address of the current scheduler. 
30 If that scheduler is determined to be down by the failover daemon, then the 
DLBA server ARPs for the VIP. 
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Panic Mode 



hash table among * sewers in the cluster ,n «. — exfeting 
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30 box static* until the prob.em can be diagnosed and repa,red. 
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Monitoring 

An ioctl() is used to dump the DLBA server Status table and to dump the Client 
IP Hash table. A daemon is written that performs the ioctl() and writes it to the 
5 socket read-only. This can be connected to a user-interface. 

Prototype 

A prototype! of the scheduler can be developed without DLBA servers. The 
10 prototype is similar to a standard TCP/IP routing based NAT load balancer. If 
Web servers are run on each machine instead of DLBA servers, the scheduler 
routes traffic to the servers and they respond directly to the client. When the 
Web servers fail, the scheduler routes around them. When the scheduler fails, 
another box picks up as the scheduler. 

15 

DLBA Server Architecture 

With respect to Fig. 6, a DLBA server 601 is essentially a proxy Web server. It is 
a standard application level Web server that takes requests from clients 603 and 
20 forwards them 604 to back end Web servers 602. The DLBA server 601 
processes the replies 605 from the back end Web servers 602 and sends the 
response directly back to the client 606. 

If incoming requests are SSL, the DLBA server 601 performs the key exchange 
25 and decrypts the request before forwarding it to the back end Web servers 602. 
The DLBA server 601 will also encrypt the response on the way out 606 for SSL 
sessions. 

The view of a DLBA server is that it accepts incoming connections to the VIP, 
30 processes them, and returns a reply from a source of the VIP. Packets to a 
DLBA server are routed by a DLBA scheduler. However, the DLBA proxy Web 
server process does not know that. 
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. on and 443 on the M\r 

Th e DLBA server Web process rS^ZZZ* with - « 
ad dress. When ^«^^^^^«.^«»* 

™. dlba ^ « - .e - r: "S^ 

the c»en, whenever i. reads a packet from the back end 

j me incoming sess,on . kept aWe > . ( ^ ^ For , given 

end wet, ~ -epiVis nex( packe( js wrwen , 0 

. , the back end Web servers does no. need to b. kept aiive but 
:5 The connection to the back end ^ ronnect , ons) . 

l„ fact, it is better to not keep the back en ^ ^ jf )h> 

Parsin g to — . 

DLBA server performs URL hash Dase 



20 requests. 



... u m ra server was a load balancer 
■~ rase in wh ch the DLBA servei wa 

Consider, * ^ A^^L. ~* «** — < ^ ^ 

for a set of Web caches. An inawn se[ver was 

machines. 

i^hq a keen alive connection to the client, 
For example, when the DLBA server keeps a keep 

30 it will: 

. read the first request for a.html 

. hash a.html to a back end Web cache 
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o open a socket and read a.html from the back end, writing it to the client as it 
reads it 

o close the socket to the back end Web cache 
o hash b.gif to a back end Web cache 
5 o open a socket and read b.gif from the back end, writing it to the client as it 
reads it 

o close the socket to the back end Web cache 
o hash c.gif to a back end Web cache 

o open a socket and read c.gif from the back end, writing it to the client as it 
10 reads it 

o close the socket to the back end Web cache 

Notice that the connection to the client is kept alive through the whole process 
even though three socket connections to the back end were performed. 



15 



20 



Since the connections to the back end are all through a switch, performance 
across them is extremely high. Keeping the connection alive to the client 
reduces the number of packets on the Internet, thereby increasing the end user 
performance. 
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The DLBA server parses incoming requests and schedules them based on a 
scheduling algorithm. The two most popular algorithms are probably . 
cookie/session id based scheduling and URL based scheduling. Cookie based 
scheduling will persistently map an individual user to a back end Web server 
25 regardless of whether they are using SSL or HTTP. The back end Web server 
only sees HTTP traffic because the DLBA server performs decryption. 

The most common URL based scheduling is hash based scheduling with either a 
consistent hash or a binary list that has the properties of a consistent hash. This 
30 is used when the system is a front end to an array of Web caches. The 
cookie/session ID based scheduling is used when the system is a front end to a 
set of application Web servers that keep state information based on a user's I 
cookie. i 



BNSDOCID: <WO O169890A1_l_> 



WO 01/69890 



PCT/USO 1/07574 



Configuration 

■ , . DLBA server is performed by the same process as the 
Configuration * • > 
5 con.gura.on o D^s*«, £ . ,„,. The same 

:::: i new — * . - °- ~- 

in the array (including the DLBA scheduler). 
10 Internal Operation 

schedules requests to back end We ^ ^ a Web 

or session ID in an HTTP get request or by hashing 
15 server number. 

* te ta hiP needs to be maintained since the 
, f the URL hash is used, then no -» ^ ^ ^ ^ 

ha sh can simply hash **^££m «, » — — " ' 

back end Web servers changes, then tn ( ^ of 

20 ftey are caches, the caches will simp* » • ^ ^ 

something ft* » *• .**«<- °< « ^ ^ scheduling so 

rrr-rr-rrrj^ 

is called "cookie injection . 
Cookie Injection 

• , ,,„n ID scheduling is implemented, cookie injection is used to 
When cookie/sesston ID scheduling v This is a novel feature of the 
map a user to a specific machine in the cluster. Th,s ,s 
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DLBA. 
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If persistent cookie scheduling is enabled, the DLBA servers scan the headers of 
outgoing requests from the Web caches looking for the Set-Cookie header item 
that is issued by the Web server to initiate a session. The specific cookie name 
5 that starts a session is configurable. 

When the DLBA server sees a Web server initiate a Set-Cookie request, it injects 
another additional name/value pair into that set cookie that identifies the server 
that the Set-Cookie came from. For example, something like "SpServer=1 0" is 
1 0 injected as an additional cookie name/value pair in the Set-Cookie response to 
the client. 



This additional cookie has the same expiration time as the session identifier so 
persistence will be maintained as long as the session is maintained. 

15 

When requests come in, the DLBA server looks for the SpServer cookie. If the 

t 

DLBA server finds the cookie, it will schedule to the server identified by the 
cookie value. Each server can have a unique integer key, defined in the 
configuration file, that the cookie value maps to. 

20 

Reviewing the process again, the DLBA server scans incoming requests for a 
SpServer cookie. If it finds one, it looks at the value and schedules the request 

* 4 

to the machine represented by the value. The DLBA server scans headers from ^ 
Web server responses for a Set-Cookie that is creating a cookie to start a " 
25 session. When it finds one, it injects a SpServer cookie into the Set-Cookie reply 
with the same attributes as the session. 

The name of the cookie the system injects should be configurable in the 
configuration file. 

30 

By storing the cookie-to-server mapping on the client, the DLBA servers do not 
have to maintain a table that maps cookies to back end Web servers. Also, since 
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the mapping will go away or last longer than the 



Status Tables 



-r «nh DLBA server maintains a server status table 701 of all 
Referring to Rg. 7, each DU*^ ^ ^ about the 

the back end Web servers. Th.s table ^ 
bacR end web serve rs . is open Connections 

Avg. Resp. Time 703, Number of Hits 7U4, an 



10 705. 



when a DLBA server tries to connect to a ^ - ^rs 

— ■ -rrm: ~ can, — * . 



that time. 



20 an overall status stable. 

Wt h re.ec, to Fig. S, the Summa. We. 

^heaCeDLBAschedule, and no, tor 

be synchronized across mach.nes s.nce * ■ *** m<ess 802 , 

25 scheduling. ^-^^tTJl^^--™- 

Number of Cookie Groups Assigned 803, Total Num 

Number of Open Connections 805. 

Using cookie section removes the P- "^^ 
30 when a server is added, ^^'f^^^^^v, 

be lni ected w«h that new machine, identified ^^ ac t Lhines, etc. 
worry about cooKie hash groups or synching state across m 
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DLBA servers are able to load balance incoming requests to the back end Web 
servers in an intelligent fashion (open connections, machine load, SNMP 
information, response time) to distribute user sessions among the back end Web 
server machines. 

The DLBA servers should have the ability to perform SSL encryption and 
decryption in hardware (using ncipher or rainbow cards) to accelerate the 
encryption/decryption process. 



10 Monitoring 




Each DLBA server will be running a daemon that has a socket open that dumps 
the state information of the tables used for scheduling and status. This can be 
the same daemon that the DLBA scheduler uses to dump its state information 
15 and the information dumped by the DLBA server should be in the same format as 

the DLBA scheduler data. \ r 

Task Description 

20 Referring to Fig, 9, the Rev Traffic Pkts module 901 receives incoming traffic 
packets into the DLBA scheduler or server. The Rev Traffic Pkts module 
distributes the packets depending on whether the destination of the packets are &<-:J^ 
to a server, or a client (response) and whether the DLBA server is also the 
scheduler. fe; 

25 

If the DLBA server is also the scheduler, packets destined for a Web server are y " ■ {?■■ 

routed through the Route/Load Balance Pkts module 906. The Route/Load 
Balance Pkts module 906 performs the routing and load balancing of the packets 
to the DLBA servers (including itself) in the array using the Client IP Hash Table 
30 912. The Route/Load balance Pkts module 906 uses the Server Status Table j ?.*■'■" 
911 to determine the MAC address to route the packet(s) to, as previously 
described. -y-' 7 
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■ tHo Rc w Traffic Pkts module 
senses trom the Web ~ ~ ^ ^ ^ PM t0 
901 and are sen, to .he Forward PM « C*nM» ^ 
Cien, module 909 torwards the '^^ m before me response is 
are encrypted by the Encrypt/Decrypt SSL 
S en,d I reotr / «o.heo«entbytheForwardPk.«oC.,ent 

„ *e — n is -^^:j;; re rrtr;^ 
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WO °"" 90 d ,e 903 aooepts pushed con, ig ura«on Morton 

The Configure System ™^<» 912 ^ Server Status Tab.e 

which is used to establish the CM IP Hash 

911. 

status of the resident server is sent 
Daemon 910. 

_ as des c,ibed above, periodically checks with |||| 
.0 The Faiiover/Synch Daemon «*« ^ „ tn e active 

* e S0hedU ' 9r ,0 de,e T^pC 9 3 ,or the MAC address o, the schedule, 
scheduler by check,ng the ARP Table becomes me new 

scheduler is taken upon by th. ^ ^ ^ ^ ^ ^ „ new 

15 Scheduler moauies 
scheduler. 

. «hP RCV Traffic Pkts module 901 are 
,„ the role of server, packets from the ^ Connection 

processed b y the Process ^ scheduler , from clients and 

20 module 907 takes the requests, routed y ^ by ^ _ 

Encrypt/Decrypt SSL moouie » 
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by the Claims included below. 
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CLAIMS 



, A process ,o, rou«ng through a load — « - "~ 

providing a plurality of load balanang sewers, 

x . hark end Web server; 

, 0 S and ,oad balances said re.es, packet to a 

ioad ba ::: «- . -~ — - ^ ^ said request 

said client. 

„ Claim 1 wherein said scheduler routes and load dances 
2. The process of uiaim 

20 client requests to itself. 

3 ^ process o, Claim 1 , further comprising the steps of: 
detecting the failure of said scheduler; and 
electing one of said load balancing servers as the 



25 



load balancing servers; and where.n sa» 



30 5. 



IOau uai»" - 

failed load balancing servers. 

, Th e process o, Claim 1. wherein said load balancing sewer schedules 
LJs h :o^endWebse.ersbasedonacooKieorsess,onlO. 
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6. The process of. Claim 1 , wherein said load balancing server uses cookie 
injection to map a client to a specific back end Web server. 

7. The process of Claim 1 , wherein said load balancing server decrypts said 
5 request packet if it is an SSL session before routing and load balancing said 

request packet to a back end Web server. 

8. The process of Claim 7, wherein said load balancing server encrypts said 
response packet if it is an SSL session before sending said response packet 

1 0 directly to said client. 

9. The process of Claim 1 , wherein said load balancing server establishes a 
connection with said client and said client keeps said connection alive with said 
load balancing server. 

15 

10. The process of Claim 9, wherein said load balancing server performs URL 
based scheduling of request packets. 

1 1 . The process of Claim 9, wherein said load balancing server performs hash 
20 scheduling of request packets. 

12. The process of Claim 1, wherein said load balancing server maintains 
persistent connections in all its paths when required; and wherein said load 
balancing server uses hash group based persistence to maintain its persistence 

25 tables. 



13. The process of Claim 1, wherein said load balancing server detects if a 
back end Web server fails; and wherein said load balancing server stops routing 
request packets to failed back end Web servers. 

30 

14. The process of Claim 1 , further comprising the step of: ' " 
providing a content delivery network; and 

f^-*- Jrr* ■ 
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are cached to improve performance. 

acro ss a networK in a computer environment, comprising: 
a plurality of load balancing servers; 
at least one back end Web server, 

Irl r — -s and ioad baiances said re.es, packet to a 
15 '° ad rl*. dancing sen,er routes and load baiances said request 

pa *:;:riTar::web — - - — 

20 said client. 

, „f Claim 16 wherein said scheduler routes and load 
17. The apparatus of Claim 10, »' 

balances client requests to itself. 
~ 18 The apparatus of Claim 16, further comprising: 

a module for electing one o, said load baiancng servers 

scheduler. 

, „ of Claim 16 wherein said scheduler detects the failure of 
other load balancing servers; and where.n 
to any failed load balancing servers. 
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20. The apparatus of Claim 16, wherein said load balancing server schedules 
sessions to back end Web servers based on a cookie or session ID. 

21. The apparatus of Claim 16, wherein said load balancing server uses 
cookie injection to map a client to a specific back end Web server. 

22. The apparatus of Claim 16, wherein said load balancing server decrypts 
said request packet if it is an SSL session before routing and load balancing said 
request packet to a back end Web server. 

23. The apparatus of Claim 22, wherein said load balancing server encrypts 
said response packet if it is an SSL session before sending said response packet 
directly to said client, 

24. The apparatus of Claim 16, wherein said load balancing server 
establishes a connection with said client and said client keeps said connection 
alive with said load balancing server. 

25. The apparatus of Claim 24, wherein said load balancing server performs 
URL based scheduling of request packets. 

26. The apparatus of Claim 24, wherein said load balancing server performs 
hash scheduling of request packets. 

27. The apparatus of Claim 16, wherein said load balancing server maintains 
persistent connections in all its paths when required; and wherein said load 
balancing server uses hash group based persistence to maintain its persistence 
tables. 

28. The apparatus of Claim 16, wherein said load balancing server detects if a 
back end Web server fails; and wherein said load balancing server stops routing 
request packets to failed back end Web servers. 
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29 . The apparatus of Claim 16, further comprising: 
a content delivery network; and 

♦ • of Cairn 29 wherein HTML pages that have modified 
30. The apparatus of Claim «"» 
URLs are cached to improve performance. 
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